{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "[View the runnable example on GitHub](https://github.com/intel-analytics/BigDL/tree/main/python/nano/tutorial/notebook/inference/pytorch/pytorch_save_and_load_openvino.ipynb)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Save and Load OpenVINO Model\n",
        "\n",
        "This example illustrates how to save and load a model accelerated by openVINO.\n",
        "\n",
        "In this example, we use a pretrained ResNet18 model. Then, by calling `trace(..., accelerator=\"openvino\")`, we can obtain a model accelarated by openVINO method provided by BigDL-Nano for inference. By calling `save(model=..., path=...)` , we could save the Nano optimized model to a folder. By calling `load(path=...)`, we could load the OpenVINO optimized model from a folder."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nbsphinx": "hidden"
      },
      "source": [
        "To conduct OpenVINO optimization through BigDL-Nano InferenceOptimizer, you need to install BigDL-Nano for PyTorch inference first. We recommend you to use [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to prepare the environment and install the following packages in a conda environment.\n",
        "\n",
        "You can create a conda environment by executing:\n",
        "\n",
        "```\n",
        "# \"nano\" is conda environment name, you can use any name you like.\n",
        "conda create -n nano python=3.7 setuptools=58.0.4\n",
        "conda activate nano\n",
        "```\n",
        "\n",
        "> 📝 **Note**: \n",
        ">\n",
        "> During your installation, there may be some warnings or errors about version, just ignore them.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "nbsphinx": "hidden"
      },
      "outputs": [],
      "source": [
        "# Necessary packages for inference accelaration using OpenVINO\n",
        "!pip install --pre --upgrade bigdl-nano[pytorch,inference]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "First, prepare model. We need to load the pretrained ResNet18 model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import torch\n",
        "from torchvision.models import resnet18\n",
        "\n",
        "model_ft = resnet18(pretrained=True)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Accelerate Inference Using OpenVINO"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from bigdl.nano.pytorch import InferenceOptimizer\n",
        "\n",
        "ov_model = InferenceOptimizer.trace(model_ft,\n",
        "                                    accelerator=\"openvino\",\n",
        "                                    input_sample=torch.rand(1, 3, 224, 224))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Save Optimized Model\n",
        "The saved model files will be saved at \"./optimized_model_ov\" directory.\n",
        "\n",
        "There are 3 files in optimized_model_ov, users only need to take \".bin\" and \".xml\" file for further usage:\n",
        "\n",
        "* nano_model_meta.yml: meta information of the saved model checkpoint\n",
        "* ov_saved_model.bin: contains the weights and biases binary data of model\n",
        "* ov_saved_model.xml: model checkpoint for general use, describes model structure"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "InferenceOptimizer.save(ov_model, \"./optimized_model_ov\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Load the Optimized Model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "loaded_model = InferenceOptimizer.load(\"./optimized_model_ov\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> 📝 **Note**\n",
        ">\n",
        "> For a model accelerated by OpenVINO, we save the structure of its network. So, the original model is not needed when we load the optimized model."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Inference with the Loaded Model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "with InferenceOptimizer.get_context(loaded_model):\n",
        "    x = torch.rand(2, 3, 224, 224)\n",
        "    y_hat = loaded_model(x)\n",
        "    predictions = y_hat.argmax(dim=1)\n",
        "    print(predictions)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> 📚 **Related Readings**\n",
        ">\n",
        "> - [How to install BigDL-Nano](https://bigdl.readthedocs.io/en/latest/doc/Nano/Overview/install.html)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "nano-pytorch",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.15 (default, Nov 24 2022, 21:12:53) \n[GCC 11.2.0]"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "8a5edab282632443219e051e4ade2d1d5bbc671c781051bf1437897cbdfea0f1"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
